Omnidirectional camera pose estimation and projective texture mapping for photorealistic 3D virtual reality experiences

Modern applications in virtual reality require a high level of fruition of the environment as if it was real. In applications that have to deal with real scenarios, it is important to acquire both its three-dimensional (3D) structure and details to enable the users to achieve good immersive experiences. The purpose of this paper is to illustrate a method to obtain a mesh with high quality texture combining a raw 3D mesh model of the environment and 360 ° images. The main outcome is a mesh with a high level of photorealistic details. This enables both a good depth perception thanks to the mesh model and high visualization quality thanks to the 2D resolution of modern omnidirectional cameras. The fundamental step to reach this goal is the correct alignment between the 360 ° camera and the 3D mesh model. For this reason, we propose a method that embodies two steps: 1) find the 360 ° cameras pose within the current 3D environment; 2) project the high-quality 360 ° image on top of the mesh. After the method description, we outline its validation in two virtual reality scenarios, a mine and city environment, respectively, which allows us to compare the achieved results with the ground truth.</p

Object Pose Detection to Enable 3D Interaction from 2D Equirectangular Images in Mixed Reality Educational Settings

In this paper, we address the challenge of estimating the 6DoF pose of objects in 2D equirectangular images. This solution allows the transition to the objects&rsquo; 3D model from their current pose. In particular, it finds application in the educational use of 360&deg; videos, where it enhances the learning experience of students by making it more engaging and immersive due to the possible interaction with 3D virtual models. We developed a general approach usable for any object and shape. The only requirement is to have an accurate CAD model, even without textures of the item, whose pose must be estimated. The developed pipeline has two main steps: vehicle segmentation from the image background and estimation of the vehicle pose. To accomplish the first task, we used deep learning methods, while for the second, we developed a 360&deg; camera simulator in Unity to generate synthetic equirectangular images used for comparison. We conducted our tests using a miniature truck model whose CAD was at our disposal. The developed algorithm was tested using a metrological analysis applied to real data. The results showed a mean difference of 1.5&deg; with a standard deviation of 1&deg; from the ground truth data for rotations, and 1.4 cm with a standard deviation of 1.5 cm for translations over a research range of &plusmn;20&deg; and &plusmn;20 cm, respectively

Impact facility based upon high frequency two stage light-gas gun

An impact facility based upon a two-stage high-frequency light-gas gun has been developed to allow fast and low-cost hypervelocity tests. The mechanical configuration and the managing electronic system are presented. The unit is powered only by means of high-pressure gas: no explosive powder is used. The system is managed by a dedicated computer system, which acquires signals from pressure transducers and operates nine electron valves. To improve the gun reliability, the control system has been designed to carry out an automatic diagnostic procedure after each shot. To improve the gun performance and the safety, an active piston-damping procedure has been developed. In this configuration a high shot frequency has been reached (10 shots/h). Projectiles with mass between 100–300 mg have been launched up to 3–4 km/s depending on the mass. This work can also be seen as a feasibility study for a new class of high-performance, highfrequency and low-cost two-stage light-gas guns, useful for the Italian Hypervelocity Laboratory, proposed to be built at the Italian Center for Aerospatial Research (CIRA)

Adaptive-randomised self-calibration of electro-mechanical shutters for space imaging

This work describes the self-calibration of a high-precision open-loop mechanism. The self-calibration method is applied to a mechanical shutter for space applications, which was launched onboard the ESA-ROSETTA mission (launch: 2 March 2004). It is based on an adaptive \u2018model reference\u2019 and a \u2018randomised\u2019 search method which may be generalised to applications in which high performance and functionality are strongly interconnected. The method makes use of an adaptive \u2018model-reference\u2019 control approach [K.J. Astrom, B. Wittenmark, On self-tuning regulators Automatica 9 (1973) 185\u2013199 [16]; K.J. Astrom, Theory and application of adaptive control, in: Proceedings of the Eighth IFAC World Conference, Kyoto, Japan, 1981 [17]; D.E. Seborg, S.L. Shah, T.F. Edgar, Adaptive control strategies for process control, AIChE Journal 6(32) (1986) 881\u2013895 [18]] to guarantee mechanism performance. The proposed control system comprises both a deterministic adaptive part and a random-search one [K.L. Clarkson, Applications of random sampling in computational geometry, II, in: Proceedings of the Fourth Annual ACM Symposium on Computational Geometry, 1998, pp. 1\u201311 [19]; P.K. Agarwal, M. Sharir, Efficient randomised algorithms for some geometric optimisation problems, Discrete Computational Geometry 16 (1996) 317\u2013337 [15]] to guarantee shutter mechanism functionality and performance over testing lifetime (5x104 cycles)